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the mouse. Based on the output signal, the display widget (1010) such as icon, tool, scroll box, dialog box, combo 
box, is visible or faded out of view in the graphical display (1020). 

DESCRIPTION - An INDEPENDENT CLAIM is also included for graphical user interface implementing method. 
USE - For implementing graphical user interface used for computer system, with on-demand display function. 
ADVANTAGE - As widget is displayed corresponding to user's mouse touching condition, display clutter is reduced 
and available application screen area is increased, thus user experience is improved. 

DESCRIPTION OF DRAWINGS - The figures show the example of screen display showing fade-in and fade-out of 
display widget. 
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1020 Graphical display 
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(57) ABSTRACT 

A technique, specifically apparatus and accompanying 
methods, for implementing an on-demand "Tool Glass" 
based desktop user interface. The interface uses at least one 
input device capable of detecting touch. A sensed touch 
transition reflective of a user then making or breaking 
contact with the device, such by touching the device with a 
finger of a non -preferred hand or lifting his(her) finger from 
the device, causes a Tool Glass sheet to be displayed or 
dismissed. To prevent user distraction, these detected tran- 
sitions preferably initiate corresponding predefined anima- 
tion sequences that occur over preset time intervals in which 
the Tool Glass sheet either begins to fade into view as soon 
as user contact begins and then begins to fade out from view 
as soon as user contact ends. Such touch sensing can readily 
be used to provide "on-demand" display and dismissal of 
substantially any display widget, e.g., a toolbar, based on 
sensed contact between each hand of a user and a corre- 
sponding input device, such as between a preferred hand and 
a touch sensitive mouse. Through use of this interface, 
display clutter can be reduced and displayed application 
screen area increased at appropriate times during program 
execution consistent with user action and without imposing 
any significant cognitive burden on the user to do so; 
thereby, advantageously improving a "user experience". 

92 Claims, 8 Drawing Sheets 
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software could attempt to infer touch and release events by 
observing mouse motion and choosing time-out periods. 
However, such inferences will likely have an error rate 
associated with them; nonetheless, such inferences may 
prove useful in situations where a Touch Mouse is not 
available. 

Alternatively, a track ball modified to detect touch, or 
having appropriate peripherally located buttons, could be 
used in lieu of the Touch Mouse or the touchpad to provide 
on-demand sensing. Moreover, the touchpad could be 
replaced by another Touch Mouse or a conventional 
mouse — though, with a conventional mouse, mouse clicks 
or software inference (as noted above) could be used as a 
substitute to generate user events which the former would 
generate based on touch. Furthermore, a second touchpad, 
with appropriate peripheral buttons (as shown in FIG. 3), 
could be used in lieu of a Touch Mouse; hence, concurrently 
utilizing two touchpads, rather than one and a Touch Mouse, 

Moreover, a touch sensing keyboard, capable of sepa- 
rately sensing contact with each hand, in combination with 
a conventional mouse could be used in lieu of or in com- 
bination with both the Touch Mouse and the touchpad, or in 
conjunction with the Touch Mouse to provide on-demand 
display widget display and dismissal. Here, a toolbar(s) 
could completely fade-out starting when a user enters text 
through the keyboard. If one of the touch sensors on the 
keyboard were to detect that one hand left the keyboard 
while the other remained in contact with the keyboard, then 
the toolbars could begin to fade-in. As such, by the time the 
user were to grab a mouse with his(her) preferred hand, (s)he 
would begin to see the toolbar. If a Touch Mouse were used, 
then once the mouse sensed contact with the user, the 
interface could complete the fade-in of the toolbar rendering 
it completely visible. On the other hand, if the user did not 
grab the Touch Mouse within a short period of time after 
hLs(her) hand left the keyboard, the interface could then 
completely fade-out the toolbar. These touch sensors could 
also be used to uniquely detect handedness (i.e., left or right) 
of the user. Here too, non-contact proximity sensors could be 
integrated into the keyboard, in lieu of or in combination 
with contact sensors, to detect when the user's hand(s) is 
close to the keyboard but not actively engaging it. 

Furthermore, my inventive interface could be extended 
such that when an application first starts, the toolbars would 
be visible and a document would appear underneath the 
toolbars). Whenever a user releases the Touch Mouse (or 
other touch sensitive input pointing device for preferred 
hand use), a margin will appear above the document, thus 
permitting the user to perceive the top of the document. 
Doing so prevents this portion of the document from being 
covered by the toolbars)— -as might conventionally occur. 
Moreover, when the application first starts and the user is not 
contacting the Touch Mouse, the toolbars) could be dis- 
played so that the user is not surprised when the toolbar(s) 
subsequently vanishes. In this instance, the toolbars) could 
disappear only after the user first touches and then releases 
the Touch Mouse — in effect the user provides an explicit 
input event to trigger a toolbar transition and thus under- 
stands its disappearance. Where a user engages in a click- 
through-and-drag operation with a Tool Glass sheet and in 
conjunction with the Touch Mouse, the inventive interface 
could be extended such that a toolbars) will not fade back 
in until the user releases a mouse button, even if (s)he were 
to lift his(her) finger off the touchpad. If the user were still 
touching the touchpad, then the Tool Glass sheet could fade 
back in once the mouse button is released. 

In addition, user events other than touch sensing or in 
combination with touch sensing could be used to effectuate 
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an on-demand interface. These events can include, e.g.: 
mouse cursor position, mouse motion, mouse button click or 
release events, mouse wheel motion, location of a keyboard 
insertion point in a document, type or location of a currently 

5 selected object on a display screen, time-outs or time delays 
after the occurrence of a particular event. Moreover, other 
sensing technologies, apart from touch sensing, could be 
used to invoke on-demand action, such as, e.g.: galvanic 
skin response, non-contact proximity sensors, pressure 

io sensors, events from touch/proximity sensors on a keyboard, 
data from a GPS (global positioning system) receiver 
(position of the user or location of a mobile computer), video 
data from a camera, and audio data from a microphone. 
Moreover, conventional decision-theoretic approaches 

15 could be integrated with my inventive on-demand interface 
to produce additional or improved decisions as to how to 
combine multiple user events or produce reasoned decisions 
about what portions of a screen display are important and 
need to be hidden or displayed. 

20 In addition, though I have described my invention in the 
context of use with a two-handed user interface, the 
on-demand aspect of the invention is equally applicable to a 
one-handed interface where touch is used to display and/or 
dismiss a Tool Glass or other display widget, either with 

25 controlled animation or not. However, use of a Tool Glass is 
considerably less efficient through a one-handed interface, 
where its display, dismissal and manipulation are all con- 
trolled through preferred hand interaction, than with a two- 
handed one. 

30 

Although an embodiment which incorporates the teach- 
ings of the present invention has been shown and described 
in detail herein, those skilled in the art can readily devise 
many other embodiments that still utilize these teachings. 
3S I claim: 

1. Apparatus for use with a computer system for imple- 
menting a graphical user interface, the system having a 
processor and a memory, connected to the processor, for 
storing computer executable instructions therein, and a dis- 
4Q play connected to the system, the apparatus comprising: 
a first input device, connected to the system, for being 
manipulated by a dominant hand of the user, the first 
input device having a first touch sensor for generating 
a first output signal responsive to and indicative of 
45 whether the user is touching the device or not, and 
wherein the first input device is a touch sensitive mouse 
comprising, 
a palm support; and 

a palm sensor, as the first touch sensor and situated 
50 directly on the palm support, for determining 

whether a palm of the dominant hand is abutting 
against the palm support so as to generate the first 
output signal; 

a graphical display, rendered on the display and produced 
55 by the system in response to the executable 
instructions, having a first display widget situated 
thereon; and 

wherein the system, in response to the first output signal, 
fades the first display widget into or out of view on the 
60 graphical display based on whether the dominant hand 
of the user has then established or broken physical 
contact, respectively, with the first input device by 
correspondingly touching or releasing the first input 
device. 

65 2. The apparatus in claim 1 wherein the processor in 
response to the executable instructions, controls fading of 
the first display widget in response to the first output signal 
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such that upon initiation of the physical contact the first 
display widget fades into view from a first state having a first 
predefined opacity to a second state having a second pre- 
defined opacity and upon cessation of the physical contact 
the first display widget fades out from view from the second 5 
state back to the first state. 

3. The apparatus in claim 2 wherein the first and second 
predefined opacities are approximately full transparency and 
full opacity, respectively. 

4. The apparatus in claim 2 wherein the first display 3Q 
widget comprises an icon, a tool, hover text, a scroll bar, a 
window frame, a window decoration, a title bar, a floating 
tool palette, a modal dialog box, a non-modal dialog box, a 
list box, a combo box, a button, or a text entry area. 

5. The apparatus in claim 2 wherein 

the processor, in response to the executable instructions, 15 
initiates fading of the first display widget into view and 
out of view in response to corresponding transitions in 
the first output signal indicative of the dominant hand 
of the user establishing and breaking physical contact 
with the first in input device; and wherein 20 

the processor, in response to the executable instructions, 
fades the first display widget into and out of view over 
predefined first and second time intervals, respectively, 
and controls opacity of the first display widget while 
fading during each of the first and second intervals in 25 
response to predefined first and second functions, 
respectively. , 

6. The apparatus in claim 5 wherein either of the pre- 
defined first and second functions is a predefined non-linear 
function of opacity with respect to time. 30 

7. The apparatus in claim 6 wherein the non-linear func- 
tion provides slow-in/slow-out fading. 

8. The apparatus in claim 5 wherein either of the first and 
second predefined functions is a linear function. 

9. The apparatus in claim 5 wherein the first and second 35 
intervals are, in duration, approximately 0.5 seconds or less, 
and at least approximately 1 second, respectively. . 

10. The apparatus in claim 5 wherein the first and second 
predefined opacities are approximately full transparency and 
full opacity, respectively. 40 

11. The apparatus in claim 5 wherein the first display 
widget comprises an icon, a tool, hover text, a scroll bar, a 
window frame, a window decoration, a title bar, a floating 
tool palette, a modal dialog box, a non-modal dialog box, a 
list box, a combo box, a button, or a text entry area. 45 

12. The apparatus in claim 5 wherein the first function is 
different from the second function. 

13. The apparatus in claim 5 further comprising graphics 
and output circuitry, connected to the processor and the 
display, capable of implementing alpha-transparent graphic 50 
operations wherein the processor, in response to the execut- 
able instructions, issues instructions to the circuitry so as to 
change an alpha-transparency value of the first display 
widget rendered on the display for successive display frames 
occurring during the first and second intervals and so 55 
implement a controlled animation sequence which, during 
the successive frames, fades the first display widget into or 
out of view, respectively. 

14. The apparatus in claim 13 wherein the processor, in 
response to the executable instructions, varies coloration or 60 
luminance of the first display widget between successive 
ones of the frames during the animation sequence. 

15. The apparatus in claim 14 wherein either of the first 
and second predefined functions is a linear function. 

16. The apparatus in claim 14 wherein the first and second 65 
intervals are, in duration, approximately 0.5 seconds or less, 
and at least approximately 1 second, respectively. 
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17. r lhe apparatus in claim 14 wherein the first display 
widget comprises an icon, a tool, hover text, a scroll bar, a 
window frame, a window decoration, a title bar, a floating 
tool palette, a modal dialog box, a non-modal dialog box, a 
list box, a combo box, a button, or a text entry area. 

18. The apparatus in claim 13 wherein either of the 
predefined first and second functions is a predefined non- 
linear function of opacity with respect to time. 

19. The apparatus in claim 18 wherein the non-linear 
function provides slow-in/slow-out fading. 

20. Apparatus for use with a computer system for imple- 
menting a graphical user interface, the system having a 
processor and a memory, connected to the processor, for 
storing computer executable instructions therein, and a dis- 
play connected to the system, the apparatus comprising: 

a first input device, connected to the system, for being 
manipulated by a corresponding hand of the user, the 
first input device having a first touch sensor for gen- 
erating a first output signal responsive to and indicative 
of whether the user is touching the device or not; 

a graphical display, rendered on the display and produced 
by the system in response to the executable 
instructions, having a first display widget situated 
thereon; 

wherein the system, in response to the first output signal, 
fades the first display widget into or out of view on the 
graphical display based on whether the hand of the user 
has then established or broken physical contact, 
respectively, with the first input device by correspond- 
ingly touching or releasing the first input device; 

wherein the processor in response to the executable 
instructions, controls fading of the first display widget 
in response to the first output signal such that upon 
initiation of the physical contact the first display widget 
fades into view from a first state having a first pre- 
defined opacity to a second state having a second 
predefined opacity and upon cessation of the physical 
contact the first display widget fades out from view 
from the second state back to the first state; and 

wherein the processor, in response to the executable 
instructions, fades the first display widget into and out 
of view over predefined first and second time intervals, 
respectively, and controls opacity of the first display 
widget while fading during each of the first and second 
intervals in response to predefined first and second 
functions, respectively, the first function being different 
from the second function. 

21. The apparatus in claim 20 wherein the processor, in 
response to the executable instructions, initiates fading of 
the first display widget into view and out of view in response 
to corresponding transitions in the first output signal indica- 
tive of the non-preferred band of the user establishing and 
breaking physical contact with the first device. 

22. The apparatus in claim 21 wherein the first and second 
predefined opacities are approximately full transparency and 
full opacity, respectively. 

23. The apparatus in claim 21 further comprising graphics 
and output circuitry, connected to the processor and the 
display, capable of implementing alpha-transparent graphic 
operations wherein the processor, in response to the execut- 
able instructions, issues instructions to the circuitry so as to 
change an alpha-transparency value of the first display 
widget rendered on the display for successive display frames 
during the first and second intervals and so implement a 
controlled animation sequence which, during the successive 
frames, fades the first display widget into or out of view, 
respectively. 
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24. The apparatus in claim 23 wherein the processor, in 38. The apparatus in claim 37 wherein the processor, in 
response to the executable instructions, varies coloration or response to the executable instructions, initiates fading of 
luminance of the first display widget between successive the Tool Glass into view and out of view in response to 
ones of the frames during the animation sequence. corresponding transitions in the second output signal indica- 

25. The apparatus in claim 24 wherein either of the first 5 ti ve of the non-preferred hand of the user establishing and 
and second predefined functions is a linear function. breaking physical contact with the second input device. 

26. The apparatus in claim 24 wherein the first and second 39 ^ apparatus in claim 38 wherein either of the 
intervals are, in duration approximately 0.5 seconds or less, predefined third and fourth functions is a predefined non- 
and at least approximately 1 second, respectively. Unear fiinction of [{ wilh t l0 time 

27. The apparatus in claim 24 wherein the first and second - 40 {q ^ 39 wherein ^ non . linear 

F u 7^^ P™*« slow-in/slow-out fading. 

28 The apparatus in claim 24 wherein the first display " ^ apparatus in claim 38 wherein either of the third 

widget comprises an icon, a tool, hover text, a scroll bar, a an ° J 0 ™" 1 P redefi ned factions is a linear function 

window frame, a window decoration, a title bar, a floating 42 ™ c apparatus in claim 38 wherein the third and fourth 

tool palette, a modal dialog box, a non-modal dialog box, a 15 intervals are, in duration, approximately 0.5 seconds or less, 

list box, a combo box, a button, or a text entry area. and at least approximately 1 second, respectively. 

29. The apparatus in claim 24 wherein the first input 43. The apparatus in claim 38 wherein the third and fourth 
device is a touchpad, touch sensitive mouse, touch sensitive predefined opacities are approximately Oct and at least 
trackball or touch sensitive keyboard. approximately 0.7a, respectively, where 0a and la repre- 

30. The apparatus in claim 23 wherein either of the 20 sent full transparency and full opacity, respectively, 
predefined first and second functions is a predefined non- 44. The apparatus in claim 38 wherein the first display 
linear function of opacity with respect to time. widget comprises an icon, a tool, hover text, a scroll bar, a 

31. The apparatus in claim 30 wherein the non-linear window frame, a window decoration, a title bar, a floating 
function provides slow-in/slow-out fading. l00 l palette, a modal dialog box, a non-modal dialog box, a 

32. The apparatus in claim 21, further comprising: 25 y is[ boXj a comD o box, a button, or a text entry area. 

a second input device, wherein the first and second input 45, Th e apparatus in claim 38 wherein the first input 

devices are manipulated by preferred and non-preferred device is a touchpad, touch sensitive mouse, touch sensitive 

hands of the user, the second input device having a trackball or touch sensitive keyboard, 

second touch sensor for generating a second output 46. The apparatus in claim 38 further comprising graphics 

signal responsive to and indicative of the user touching 30 and output circuitry, connected to the processor and the 

the second input device or not; and display, capable of implementing alpha-transparent graphic 

a Tool Glass appearing on the graphical display and operations wherein the processor, in response to the execut- 

produced by the system in response to the executable able instructions, issues instructions to the circuitry so as to 

instructions; and change an alpha-transparency value of the Tool Glass ren- 

wherein the processor, in response to the second output 35 dered on the display for successive display frames occurring 

signal, controls fading of the Tool Glass such that, during the third and fourth intervals and so implement a 

when the non-preferred hand touches the second input controlled animation sequence which, during the successive 

device, the Tool Glass fades into view from a substan- frames, fades the Tool Glass into or out of view, respectively, 

tially invisible state having a third predefined opacity to 47. The apparatus in claim 46 wherein the processor, in 

a semi-transparent state having a fourth predefined 40 response to the executable instructions, varies coloration or 

opacity and, when the non-preferred hand releases the luminance of the Tool Glass between successive ones of the 

second input device, the Tool Glass fades out of view frames during the animation sequence, 

from the semi-transparent state back to the substantially 48. The apparatus in claim 47 wherein either of the third 

invisible stale. and fourth predefined functions is a linear function. 

33. The apparatus in claim 32 wherein each of the first and 45 49. The apparatus in claim 47 wherein the third and fourth 
second touch sensors produces a one-bit output signal. intervals are, in duration, approximately 0.5 seconds or less, 

34. The apparatus in claim 32 wherein the second input and at least approximately 1 second, respectively, 
device is a touchpad, touch sensitive mouse, touch sensitive 50. The apparatus in claim 47 wherein the third and fourth 
trackball or other touch sensitive input pointing device. predefined opacities are approximately 0a and at least 

35. The apparatus in claim 34 wherein the first input 50 approximately 0.7a, respectively, where 0a and la repre- 
device is a touchpad, touch sensitive mouse, touch sensitive sent full transparency and full opacity, respectively, 
trackball or touch sensitive keyboard. 51. The apparatus in claim 47 wherein the first display 

36. The apparatus in claim 34 wherein if both the Tool widget comprises an icon, a tool, hover text, a scroll bar, a 
Glass and the first display widget are simultaneously being window frame, a window decoration, a title bar, a floating 
rendered on the display and the physical contact ceases 55 tool palette, a modal dialog box, a non-modal dialog box, a 
between the preferred hand and the first input device, the list box, a combo box, a button, or a text entry area, 
processor, in response to the executable instructions and 52. The apparatus in claim 47 wherein the first input 
which specific widgets arc then being displayed, an opera- device is a touchpad, touch sensitive mouse, touch sensitive 
tion then occurring or a relative display prioritization among trackball or touch sensitive keyboard. 

widgets that could be simultaneously displayed, does not 60 53. The apparatus in claim 46 wherein either of the 
fade the first display widget from view. predefined third and fourth functions is a predefined non- 
37. The apparatus in claim 34 wherein the processor, in linear function of opacity with respect to time, 
response to the executable instructions, fades the Tool Glass 54. The apparatus io claim 53 wherein the non-linear 
into and out of view over predefined third and fourth time function provides slow-in/slow-out fading, 
intervals, respectively, and controls opacity of the Tool Glass 65 55. The apparatus in claim 20 wherein either of the 
while fading during each of the third and fourth intervals in predefined first and second functions is a predefined non- 
response to predefined third and fourth functions. linear function of opacity with respect to time. 
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56. The apparatus in claim 55 wherein the non-linear 
function provides slow-in/slow-out fading. 

57. The apparatus in claim 20 wherein either of the first 
and second predefined functions is a linear function. 

58. The apparatus in claim 20, wherein the first and 5 
second intervals are, in duration, approximately 0.5 seconds 
or less, and at least approximately 1 second, respectively. 

59. The apparatus in claim 20 wherein the first display 
widget comprises hover text or a text entry area. 

60. The apparatus in claim 20 wherein the first input 10 
device is a touch sensitive keyboard, 

61. A method for implementing a graphical user interface 
for use with a computer system, the system having a 
processor and a memory, connected to the processor, for 
storing computer executable instructions therein, and a dis- 15 
play connected to the system, the method comprising the 
steps of: 

manipulating a first input device, connected to the system, 
by a preferred hand of the 
user, the first input device having a first touch sensor for 20 

generating a first output signal 
responsive to and indicative of whether the user is touching 

the device or not; 

rendering a graphical display, on the display and produced 
by the system in response to the executable 25 
instructions, having a first display widget situated, 
thereon; 

fading, in response to the first output signal, the first 
display widget into or out of view on the graphical 
display based on whether the hand of the user has then 30 
established or broken physical contact, respectively, 
with the first device by correspondingly touching or 
releasing the first device; 

manipulating a second input device, by a non-preferred 35 
hand of the user, respectively, the second input device 
having a second touch sensor for generating a second 
output signal responsive to and indicative of the user 
touching the second input device or not; 

rendering a Tool Glass on the graphical display and 40 
produced by the system in response to the executable 
instructions; and 

in response to the second output signal, fading the Tool 
Glass such that, when the non-preferred hand touches 
the second input device, the Tool Glass fades into view 45 
from a substantially invisible state having a third pre- 
defined opacity to a semi-transparent state having a 
fourth predefined opacity and, when the non-preferred 
hand releases the second input device, fading the Tool 
Glass fades out of view from the semi-transparent state 50 
back to the substantially invisible state. 

62. The method in claim 61 further comprising the step of 
controlling fading of the first display widget in response to 
the first output signal such that upon initiation of the 
physical contact the first display widget fades into view from 55 
a first state having a first predefined opacity to a second state 
having a second predefined opacity and upon cessation of 
the physical contact the first display widget fades out from 
view from the second slate back to the first state. 

63. The method in claim 62 wherein the first and second 60 
predefined opacities are approximately full transparency and 
full opacity, respectively. 

64. The method in claim 62 further comprising the steps 

of: 

fading the first display widget into and out of view over 65 
predefined first and second time intervals, respectively; 
and 
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controlling opacity of the first display widget while fading 
during each of the first and second intervals in response 
to predefined first and second functions, respectively. 

65. The method in claim 64 further comprising the step of 
initiating fading of the first display widget into view and out 
of view in response to corresponding transitions in the first 
output signal indicative of the non-preferred hand of the user 
establishing and breaking physical contact with the first 
device. 

66. The method in claim 65 wherein either of the pre- 
defined first and second functions is a predefined non-linear 
function of opacity with respect to time. 

67. The method in claim 66 wherein the non-linear 
function provides slow-in/slow-out fading. 

68. The method in claim 65 wherein either of the first and 
second predefined functions is a linear function. 

69. The method in claim 65 wherein the first and second 
intervals are, in duration, approximately 0.5 seconds or less, 
and at least approximately 1 second, respectively. 

70. The method in claim 65 wherein the first and second 
predefined opacities arc approximately full transparency and 
full opacity, respectively. 

71. The method in claim 65 further comprising the step of 
changing an alpha-transparency value of the first display 
widget rendered on the display for successive display frames 
during the first and second intervals and so implement a 
controlled animation sequence which, during the successive 
frames, fades the first display widget into or out of view, 
respectively. 

72. The method in claim 71 further comprising the step of 
varying coloration or luminance of the first display widget 
between successive ones of the frames during the animation 
sequence. 

73. The method in claim 72 wherein either of the first and 
second predefined functions is a linear function. 

74. The method in claim 72 wherein the first and second 
intervals are, in duration, approximately 0.5 seconds or less, 
and at least approximately 1 second, respectively. 

75. The method in claim 72 wherein the first and second 
predefined opacities are approximately full transparency and 
full opacity, respectively. 

76. The method in claim 71 wherein either of the pre- 
defined first and second functions is a predefined non-linear 
function of opacity with respect to lime. 

77. The method in claim 76 wherein the non-linear 
function provides slow- in/slow-out fading. 

78. The method in claim 65, further comprising the step, 
if both the Tool Glass and the first display widget are 
simultaneously being rendered on the display and the physi- 
cal contact ceases between the preferred hand and the first 
input device, of not fading the first display widget from view 
in response to which specific widgets are then being 
displayed, an operation then occurring or a relative display 
prioritization among widgets that could be simultaneously 
displayed. 

79. The method in claim 65 further comprising the steps 
of fading the Tool Glass into and out of view over predefined 
third and fourth time intervals, respectively, and controlling 
opacity of the Tool Glass while fading during each of the 
third and fourth intervals in response to predefined third and 
fourth functions. 

80. The method in claim 79 further comprising the step of 
initialing fading of the Tool Glass into view and out of view 
in response to corresponding transitions in the second output 
signal indicative of the non-preferred hand of the user 
establishing and breaking physical contact with the second 
input device. 
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81. The method in claim 80 wherein either of the pre- 
defined third and fourth functions is a predefined non-linear 
function of opacity with respect to time. 

82. The method in claim 81 wherein the non-linear 
function provides slow-in/slow-out fading. 

83. The method in claim 80 wherein either of the third and 
fourth predefined functions is a linear function. 

84. The method in claim 80 wherein the third and fourth 
intervals are, in duration, approximately 0.5 seconds or less, 
and at least approximately 1 second, respectively. 

85. The method in claim 80 wherein the third and fourth 
predefined opacities are approximately 0a and at least 
approximately 0.7a, respectively, where 0a and la repre- 
sent full transparency and full opacity, respectively. 

86. The method in claim 80 further comprising the step of 
changing an alpha-transparency value of the Tool Glass 
rendered on the display for successive display frames occur- 
ring during the third and fourth intervals and so implement 
a controlled animation sequence which, during the succes- 
sive frames, fades the Tool Glass into or out of view, 
respectively. 
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87. The method in claim 86 further comprising the step of 
varying coloration or luminance of the Tool Glass between 
successive ones of the frames during the animation 
sequence. 

88. The method in claim 87 wherein either of the third and 
fourth predefined functions is a linear function. 

89. The method in claim 87 wherein the third and fourth 
intervals are, in duration, approximately 0.5 seconds or less, 
and at least approximately 1 second, respectively. 

90. The method in claim 87 wherein the third and fourth 
predefined opacities are approximately 0a and at least 
approximately 0.7a, respectively, where 0a and la repre- 
sent full transparency and full opacity, respectively. 

91. The method in claim 86 wherein either of the pre- 
defined third and fourth functions is a predefined non-linear 
function of opacity with respect to time. 

92. The method in claim 91 wherein the non-linear 
function provides slow-in/slow-out fading. 
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B. method of administration of the attention performance test; 

C. computer readable medium with the program 
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computer. The brain's attention span level can be measured physiologically through a reaction time response with 
high level vigilance. The stress acting on brain can also be evaluated day to day. 
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[57] ABSTRACT 

An attention performance test uses a visual display, a test 
subject input device, a memory and a programmed processor 
coupled to each of the above to administer a test to a test 
subject. A test target, which for example can be a word 
colored in a first color but meaning another color, is 
attempted to be matched by the test subject to one of a scries 
of sequentially presented visual response targets. In one 
embodiment, a nonvisual test stimulus is also provided to the 
test subject, the test subject having previously been 
instructed to respond to the nonvisual test stimulus (such as 
a beep) in a particular way. The test also measures the 
response latency of the test subject, i.e., whether the test 
subject has assumed a correct position for entering a 
response. The responses from the test subject are recorded 
and may be compared with data derived from a test subject 
database in assessing whether an attention deficit disorder 
exists. 
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screen. The technician informs the patient that the 
"beep" sound may come with a visual word target on 
the top half of the screen, or with no word, and they are 
to push the LRM buttons RLS, LLS whenever there is 
a "beep" sound, no matter what the patient sees on the 
top half of the screen. 

17. The technician programs the first target from the 
"tutorial program" and observes if the patient can react 
correctly to the target stimuli and the target responses. 
The technician may "coach" the patient at the begin- 
ning as the technician advances the targets. The patient 
should be able to react correctly by himself or herself 
after 2-10 targets. 

18. The technician enters the "trial program" of the 
tutorial screen (not shown) and select the "auditory 
alone" test type at 148 (FIG. 13). The technician 
informs the patient that the technician is going to let the 
patients try to perform the "beep" sound part of the test 
aione without the visual targets and see if the patient 
can respond and react without any help or assistance for 
about two minutes. The technician proceeds to run the 
auditory test. 

19. Next, the technician enters the "trial program" and 
selects a "Both" test type at 148 (FIG. 13), which 
generates both visual and auditory stimuli. The tech- 
nician informs the patient that the technician is going to 
let the patient try to perform the test with both the 
sound beeps and the visual word/color targets together 
by himself or herself without any "coaching" or help 
for about two minutes. This is a "practice" run to see if 
they understand the test. The technician proceeds to run 
the program and observes the patient's responses. The 
technician or examiner should be still, quiet and 
slightly behind the patient and slightly outside the 
patient's visual field. 

20. The technician computes the trial results from items 
18 and 19 using the analysis module (FIG. 26). If the 
patient scores 70% or better on the trials, the patient can 
proceed to the finalized test program. If the patient's 
scores are below 70% the technician repeats another 
two minute trial run and reviews with the patient the 
general instructions and previous mistakes the patient 
may have made on the first trial run. 

21. The technician enters the "test program" at 124 (FIG. 
11) and selects "Both" (at 148 in FIG. 13) and the 45 
duration of the test desired at 152 (6, 12, 18 minutes). 
The technician informs the patient that the technician is 
now going to run the test which will last a longer time, 
mentioning the specific time duration. The patient is to 
try to do the best he or she can and not to get upset or 
angry if he or she makes an error or mistake. The 
technician tells the patient that it is normal and com- 
mon to miss a few targets or a make a few mistakes, and 
encourages the patient to concentrate on what they are 
doing and pay equal attention to both the word/color 
matches and the "beeps". The technician makes sure 
that the patient understands that the patient is to push 
the LRM 50 as soon and as fast as he or she can when 

he or she see or hear the appropriate target response. 
The technician encourages the patient to be thinking of $ 0 
what he or she should be doing at all tunes during the 
test. 

22. When running the test program, the examiner should 
remain quiet, still and slightly out of the visual field of 
the patient. 

23. After the test is completed, the examiner compliments 
the patient on his or her performance. The examiner 
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enters the "analysis" module (FIG. 26) arid types the 
patient's code to retrieve the log result data. The 
examiner then prints data analysis for the file and for 
review. 

In summary, the present invention provides an automated 
and objectively uniform test method in which the test subject 
is asked to match one of a series visual response targets to 
an initially presented visual test target. In a preferred 
embodiment, the initial test target is a word presented in a 
particular hue but having a meaning not necessarily associ- 
ated with the hue. The test subject is then asked to match 
colors of one of a series of sequentially presented visual 
response targets to the color of the test target. Preferably a 
nonvisual test subject stimulus, for example an auditory 
target, is also presented during selected ones of these 
sequences as a further distraction. Finally, the present inven- 
tion preferably includes a method of measuring the response 
latency of the test subject during the administration of the 
test. 

While certain embodiments of the invention have been 
described in the above detailed description and illustrated in 
the appended drawings, the invention is not limited thereto 
but only by the scope and spirit of the appended claims. 

What is claimed is: 

1. Apparatus for administering an attention performance 
test, comprising: 
storage means for storing a plurality of responses by a test 

subject to an attention performance test; 
a computing device having a memory; 
a color display coupled to said computing device for 

displaying a plurality of colored visual word targets and 

a plurality of colored visual response targets in 

response to said computing device; 
a test subject input device coupled to said processor to 

communicate a test subject response to said processor; 

and 

a computer program, for execution by the computing 
device, for administering an attention performance test, 
said computer program including: 

first means for presenting for a predetermined period of 
time a first predetermined visual word target on said 
color display to the test subject, a color of the word 
target being predetermined independently of the mean- 
ing of the word target, the word target being a word for 
a predetermined color, at least some of the meanings of 
the word targets respectively denoting a color other 
than the color in which the word target is displayed; 

second means for sequentially presenting, after the first 
means has presented the predetermined visual word 
target, for predetermined periods of time, a plurality of 
predetermined visual response targets on the display, at 
least one of said visual response targets having a color 
which matches the color of the last presented visual 
word target; 

third means for monitoring, during the sequential presen- 
tation by the second means of the predetermined visual 
response targets, the test subject input device; 

fourth means for recording on said memory means, during 
the sequential presentation by the second means of the 
predetermined visual work targets, any response input 
by the test subject, said first, second, third and fourth 
means, for each of a plurality of next visual word 
targets, respectively presenting the next visual word 
target, sequentially presenting a plurality of visual 
response targets, monitoring the test input device and 
recording responses; and 
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fifth means for outputting at least to the storage means the 
recorded responses to the visual response targets. 

2. The apparatus of claim 1, wherein said storage means 
is a hard disk. 

3. The apparatus of claim 1, wherein said computer 5 
program includes sixth means for determining whether the 
response of the subject was correct. 

4. The apparatus of claim 3, wherein said computer 
program includes seventh means for storing an indicium of 
a correct response. 

5. A computer software product that includes a medium 
readable by a computer, the medium having stored thereon: 

first means for causing a computer to present, on a color 
display connected to the computer, for a predetermined 
period of time, a first predetermined visual word target 
on a color display to a test subject, a color of the word 15 
target being predetermined independently of the mean- 
ing of the word target, the word target being a word for 
a predetermined color, at least some of the meanings of 
the word targets respectively denoting a color other 
than the colors in which the respective word targets are 20 
displayed; 

second means for causing the computer to sequentially 
present, after the computer has presented the first 
predetermined visual word target, for predetermined 
periods of time, a plurality of predetermined visual 25 
response targets on the display, at least one of said 
visual response targets having a color which matches 
the color of the last presented visual word target; 

third means for causing the computer to monitor, during 
the time that the computer is sequentially presenting 30 
visual response targets, a test subject input device 
coupled to the computer; 

fourth means for causing the computer to record on a 
memory of the computer, during the time that the 
computer is sequentially presenting visual response 35 
target, any response input by the test subject, the first, 
second, third and fourth means, for each of a plurality 
of next visual word targets, respectively causing the 
computer to present a visual word target, to sequen- 
tially present visual response targets, to monitor the test 4Q 
subject input device and to record any response by the 
subject; and 

fifth means for causing the computer to output at least to 
the memory the recorded responses to the visual 
response targets. 45 

6. A method for administering an attention performance 
test, comprising the steps of: 

(a) loading into a processor instructions from a prere- 
corded computer program, the processor controlling a 
visual color display, a test subject input device and a 50 
memory all coupled to the processor to administer the 
test to the subject; 

(b) presenting for a predetermined period of time a first 
predetermined visual word target on the color display 

to the test subject, a color of the word target being 55 
predetermined independently of the meaning of the 
word target, the word target being a word for a prede- 
termined color, at least some of the meanings of the 
word targets respectively denoting colors other than the 
colors in which the respective word targets are dis- 60 
played; 

(c) after step (b), sequentially presenting, for predeter- 
mined periods of time, a plurality of predetermined 
visual response targets on the display, at least one of 
said visual response targets having a color which 65 
matches the color of the last presented visual word 
target: 
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(d) during step (c), using the processor to monitor the test 
subject input device; 

(e) during step (d), using the memory to record any 
response input by the test subject; 

(f) repeating steps (b)-{e) for each of a plurality of next 
visual word targets; and 

(g) outputting at least to the memory the recorded 
responses to the visual response targets. 

7. Apparatus for administering an attention performance 
test, comprising: 

memory means for storing an attention performance test 
computer program and a plurality of responses to the 
test of a test subject; 

a processor coupled to said memory means for retrieving 
instructions therefrom, said processor operating in 
response to said instructions; 

a display coupled to said processor for displaying a 
plurality of visual test targets and a plurality of visual 
response targets in response to said processor; and 

an input device controlled by a test subject, said input 
device coupled to said processor and communicating a 
response to said processor from the test subject, at least 
one sensor on the input device detecting whether the 
subject physically is in a position to select a response 
target in response to the visual test target; 

said processor, said memory means, said display and said 
input device administering an attention performance 
test having each of the following steps: 

(a) presenting for a predetermined period of time a first 
predetermined visual test target on said display to the 
test subject; 

(b) after step (a), sequentially presenting, for predeter- 
mined periods of time, a plurality of predetermined 
visual response targets on the display; 

(c) during step (b), using the processor to monitor the 
test subject input device; 

(d) during step (c), recording on said memory means 
any response input by the test subject and whether 
the subject is in the correct position for responding; 

(e) repeating steps (a)-(d) f° r eacn of a plurality of next 
visual word targets; and 

(f) outputting at least to the memory means the 
recorded responses to the visual response targets and 
data concerning whether the test subject was in 
position to respond. 

8. The apparatus of claim 7, wherein said memory means 
is a hard disk. 

9. The apparatus of claim 7, wherein said processor, said 
input device and said memory means are responsive to the 
computer program to determine whether the response of the 
subject was correct. 

10. The apparatus of claim 9, wherein said processor and 
said memory means are responsive to said computer pro- 
gram to store an indicium of a correct response. 

11. The apparatus of claim 7, wherein said test subject 
input device is adapted to be held by the test subject, said 
input device including means to detect whether the hands of 
the test subject are in the correct position for inputting a 
response. 

12. A computer-readable medium on which is prerecorded 
a computer program which, in conjunction with a processor, 
a memory coupled to the processor, a test-subject-controlled 
input device coupled to the processor and a display coupled 
to the processor, administers an attention performance test to 
the test subject, the computer program including: 

first program code means for causing the processor to 
present, for a predetermined period of time, a first 
predetermined visual test target on the display to the 
lest subject; 
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second program code means for causing the processor to 
sequentially present, for predetermined periods of time 
after the processor presents the first predetermined 
visual test target, a plurality of predetermined visual 
response targets on the display, at least one of said 5 
visual response targets being a correct choice; 

third program code means for causing the processor to 
monitor the test subject input device during the sequen- 
tial presentation by the processor of the visual response 
targets to detect any response by the test subject; 30 

fourth program code means for causing the processor and 
the test subject input device to monitor whether the test 
subject is in the correct position for responding during 
the sequential presentation by the processor of the 
visual response targets; 15 

fifth program code means for causing the processor to 
record, during the sequential presentation by the pro- 
cessor of the visual response targets, on said memory 
any response input by the test subject and to record the 2Q 
times when the test subject is in a correct position for 
response, the first, second, third, fourth and fifth pro- 
gram code means causing the processor, for each of a 
plurality of next visual word targets, to respectively 
present a visual word target, sequentially present a 25 
plurality of visual response targets, monitor whether 
the test subject responds, monitor whether the test 
subject is in the correct position, and record any 
response by the test subject and the times during which 
the test subject is in the correct position for response; 3Q 
and 

sixth program code means for causing the processor to 
output, at least to the memory, the recorded responses 
to the visual response targets and the times during 
which the test subject was in a correct position to 35 
respond. 

13. A method for administering an attention performance 
lest, comprising the steps of: 

(a) providing to a processor a set of instructions of a 
computer program, the processor controlling a visual 40 
display, a test subject input device and a memory all 
coupled to the processor, the processor administering 
the test to the subject according to the instructions; 

(b) presenting for a predetermined period of time a first 
predetermined visual test target on the display to the 45 
test subject; 

(c) after step (b), sequentially presenting, for predeter- 
mined periods of time, a plurality of predetermined 
visual response targets on the display, at least one of 
said visual response targets being a correct choice; 50 

(d) during step (c), using the processor to monitor the test 
subject input device tc determine if the test subject has 
selected one of the presented visual response targets; 

(e) during step (c), using the process or an d th e test 55 
subject input device to monitor whether the test subject 

is in a correct position to respond; 

(f) during steps (d) and (e), using the memory to record . 
any response input by the test subject and times during 
which the test subject was in the correct position to 60 
respond; 

(g) repeating steps (b)-(f) for each of a plurality of next 
visual word targets; and 

(h) outputting at least to the memory the recorded 
responses to the visual response targets and the times 65 
during which the test subject was in the correct position 

to respond. 
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14. Apparatus for administering an attention performance 
test, comprising: 

memory means on which is stored an attention perfor- 
mance test computer program and for storing a plurality 
of responses to the test of a test subject; 
a processor coupled to the memory means for retrieving 
instructions therefrom, said processor operating in 
response to said instructions; 
a display coupled to the processor for displaying a plu- 
rality of visual stimulus targets and a plurality of visual 
response targets in response to the processor, each of 
the visual stimulus targets having first and second 
attributes perceptible to the test subject, at least some of 
the visual stimulus targets having first attributes which 
are contradictory to their respective second attributes, 
the visual response targets having at least first attributes 
perceptible to the test subject; and 
an input device controlled by the test subject, said input 
device coupled to the processor and communicating a 
response of the test subject to the processor; 
the computer program including: 
first means for causing the processor to present, for a 
predetermined period of time, a first predetermined one 
of said visual stimulus targets on the display to the test 
subject; 

second means for causing the processor to sequentially 
present, for predetermined periods of time after the 
processor presents the first predetermined one of said 
visual stimulus targets, a plurality of predetermined 
visual response targets on the display, at least one of 
said visual response targets having a first attribute 
which matches the first attribute of the last presented 
visual stimulus target; 
third means for causing the processor to monitor the test 
subject input device during the presentation of the 
visual response targets; 
fourth means for causing the processor to record on the 
memory means any response input by the test subject 
during the presentation of the visual response targets, 
the first, second third and fourth means, for each of a 
plurality of next ones of said visual stimulus targets, 
respectively causing the processor to present one of 
said visual stimulus targets, to present visual response 
targets, to monitor the test subject input device and to 
record any response of the test subject on, the memory 
means; and 

fifth means for causing the processor to output, at least to 
the memory means, the recorded responses to the visual 
response targets. 

15. The apparatus of claim 14, wherein the memory 
means is a hard disk. 

16. The apparatus of claim 14, wherein said computer 
program includes sixth means for causing the processor to 
determine whether the response of the subject was correct. 

17. The apparatus of claim 16, wherein said computer 
program includes sixth means for causing the processor to 
store an indicium of a correct response on the memory 
means. 

18. The apparatus of claim 14, wherein the display is a 
color display, the second attribute of each predetermined 
visual stimulus target being a word for a predetermined 
color, the first attribute of each predetermined visual stimu- 
lus target being the color in which the word is displayed. 

19. The apparatus of claim 14, wherein each visual 
response target has a second attribute perceptible to the test 
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subject, at least some of the visual response targets having 
second attributes which are contradictory to the respective 
first attributes thereof. 

20. The apparatus of claim 19, wherein the display is a 
color display, the first attribute of each visual response target 5 
being a word for a predetermined color, the second attribute 
of each visual response target being the actual color in which 
the word is displayed. 

21. A medium on which is prerecorded a computer 
program which, in conjunction with a processor, a memory 10 
coupled to the processor, a test-subject-controlled input 
device coupled to the processor and a display coupled to the 
processor, administers an attention performance test to the 
test subject, the computer program including: 

first means for causing the processor to present, for a 15 
predetermined period of time, a first of a plurality of 
predetermined visual stimulus targets on the display to 
the test subject, each of the visual stimulus targets 
having individually predetermined first and second 
attributes perceptible to the test subject, at least some of 20 
the visual stimulus targets having second attributes 
which are contradictory to their respective first 
attributes; 

second means for causing the processor to sequentially 
present, after the processor presents the first prcdeter- 25 
mined visual stimulus target, for predetermined periods 
of time, a plurality of predetermined visual response 
targets on the display, each of the visual response 
targets having at least a first attribute perceptible to the 
test subject, at least one of said visual response targets 30 
having a first attribute which matches the first attribute 
of the last presented visual stimulus target; 

third means for causing the processor to monitor the test 
subject input device during the presentation of the 35 
visual response targets; 

fourth means for causing the processor to record, during 
the presentation of the visual response targets, on said 
memory any response input by the test subject, the first, 
second, third and fourth means respectively causing the 40 
processor, for each of a plurality of next visual stimulus 
targets, to present a visual stimulus target, to sequen- 
tially present a plurality of visual response targets, to 
monitor the test subject input device and to record any 
response by the test subject; and 45 

fifth means for causing the processor to output at least to 
the memory the recorded responses to the visual 
response targets. 

22. The medium of claim 21, wherein the display is a 
color display, the second attribute of each visual stimulus 50 
target being a word denoting a color, the first attribute of 
each visual stimulus target being a color in which the word 

is displayed. 

23. The medium of claim 21, wherein each visual 
response target has a second attribute perceptible to the test 55 
subject, the second attributes of at least some of the visual 
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response targets being contradictory to the respective first 
attributes of the visual response targets. 

24. The medium of claim 23, wherein the display is a 
color display, the first attribute of each visual response target 
being a word for a color, the second attribute of each visual 
response target being the color in which the word is dis- 
played. 

25. A method for administering an attention performance 
test, comprising the steps of: 

(a) loading into a processor instructions from a prere- 
corded computer program, the processor controlling a 
visual display, a test subject input device and a memory 
all coupled to the processor to administer the test to the 
subject; 

(b) presenting for a predetermined period of time a first 
one of a plurality of predetermined visual stimulus 
targets on the display to the test subject, each visual 
stimulus target having first and second attributes per- 
ceptible to the test subject, at least some of the visual 
stimulus targets having first attributes which arc con- 
tradictory to their respective second attributes; 

(c) after step (b), sequentially presenting, for predeter- 
mined periods of time, a plurality of predetermined 
visual response targets on the display, each visual 
response target having a first attribute which is percep- 
tible to the test subject, at least one of said visual 
response targets having a first attribute which matches 
the first attribute of the last presented visual stimulus 
target; 

(d) during step (c), using the processor to monitor the lest 
subject input device; 

(e) during step (d), using the memory to record any 
response input by the test subject; 

(f) repeating steps (b)-(e) for each of a plurality of next 
visual stimulus targets; and 

(g) outputting at least to the memory the recorded 
responses to the visual response targets. 

26. The method of claim 25, wherein the visual display is 
a color display, the second attribute of each predetermined 
visual stimulus target being a word for a predetermined 
color, the first attribute of each predetermined visual stimu- 
lus target being the color in which the word is displayed. 

27. The method of claim 25, wherein each visual response 
target has a second attribute perceptible by the test subject, 
at least some of the visual response targets having second 
attributes which are contradictory to the respective first 
attributes thereof. 

28. The method of claim 27, wherein the display is a color 
display, the first attribute of each visual response target 
being a word for a predetermined color, the second attribute 
of each visual response target being the actual color in which 
the word is displayed. 

***** 
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Alerting Abstract WO Al 

NOVELTY - Study materials are selected from a memory (108) and then presented to student through presenter 
(106). A nonintrusive sensor (110) monitors students concentration, relative behavior from facial expression 
periodically. The output of sensor is analyzed based on a set of rules and the concentration level is indicated based 
on analysis. 

DESCRIPTION - Based on the analysis result, the students concentration loss is prevented by asking questions to 
students unexpectedly. 

USE - For teaching subjects to students according to their concentration level through computer. 
ADVANTAGE - Ensures concentration of students on study materials by providing timely break and presenting 
questions unexpectedly to students. Monitors students concentration .without any intrusion on teaching and 
provides correct indication on concentration level. 

DESCRIPTION OF DRAWINGS - The figure shows the block diagram of computer aided learning system. 
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1 10 Nonintrusive sensor 
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Alerting Abstract US Al 

NOVELTY - The study materials on a subject are presented to a student through a presenter. The student's 
concentration-sensitive behavior is automatically monitored through a sensor when the student is studying the 
material. The monitored result is analyzed based on the rules and an indication is provided on the student's 
concentration level. 

DESCRIPTION - An INDEPENDENT CLAIM is included for computer-aided educational system. 

USE - For considering student's concentration level in teaching the students through computer at home and 

schools. 

ADVANTAGE - The sensor automatically monitors more than once, the students concentration-sensitive behavior 
while the student is working on the study material thereby, increases the accuracy in determining the student's 
concentration level. The presenter can change the presenting style accordingly such as by reducing the speed of 
presentation through increasing the line spacing of the text/size of the image. 

DESCRIPTION OF DRAWINGS - The figure shows the flowchart illustrating the process of computer-aided 
educational system. 
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Alerting Abstract US A 

NOVELTY - The system has a controller configured to set a duration of time users to communicate so as to allow 
them to work on materials on the subject. A dialogue session for users is started to communicate in an area related to 
the subject. The dialogue session is stopped approximately at or before the end of the duration of time. 
DESCRIPTION - A monitoring apparatus configured to monitor user's input to the system during the dialogue 
session so as to have the monitored input available for analysis. Based on the analysis, the controller guides user 
back to the subject in the dialogue session when one or more users have been distracted from the subject for a 
duration of time, so that the dialogue session provides an interactive environment to help the users learn. An 
INDEPENDENT CLAIM is included for a computer aided learning method. 
USE - For group of users working on subject. 

ADVANTAGE - Monitors and analyses users' inputs such as when they are interacting. 

DESCRIPTION OF DRAWINGS - The figure shows a physical implementation for an embodiment of the invention. 
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